Current research suggests that using multiple can improve affect detection accuracy. Combining facial expression and physiological signals is one of the most common approaches in multimodal affect detection. Several methods and devices have been proposed for measuring physiological signals with simplicity and have been used widely in affective computing applications. Out of the various approaches, contact-less sensors which can measure physiological signals remotely are more desirable for everyday use and naturalistic applications. In this paper we proposed a novel fusion model for affect detection, which combines facial expression features and heart rate using a single video recording sensor. To our knowledge this is the first attempt to use physiological sensor remotely for affect detection. Results suggest that fusion of these features (facial expression and heart rate) can improve the accuracy of affect detection systems.